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ABSTRACT 

The effects of violation of the assumption of 
homogeneity of regression on the Type I error rate and on the power 
of analysis of covariance (ANCOVA) were investigated. The data 
situations included in the study involved two groups with one 
covariate and one criterion, with varying equal and unequal group 
sizes, and varying degrees of violation of the assumption of 
homogeneity of regression. Results indicate that ANCOVA appeared 
robust to the violation of the assumption of homogeneity of 
regression when group sizes were equal; the technique appeared not to 
be robust for unegual group sizes. For egual group sizes and all 
slope combinations, the empirical alpha levels wer^ near the 
corresponding nominal alpha levels. For unequal group sizes and 
unegual regression slopes, hovever, large discrepancies were observed 
between toe empirical alpha levels and the corresponding nominal^ 
alpha levels. Results also indicated that the power of ANCO¥A was not 
severely altered by heterogeneous regression slopes as long as the 
group sizes were equal. (Author) 
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SUMMARY 



Title: AN EMPIRICAL INVESTIGATION OF THE EFFECTS OF 
HETEROGENEOUS REGRESSION SLOPES IN ANALYSIS 
OF COVARIANCE 

The effects of violation of the assumption of homoge- 
neity of regression on the Type I error rate and on the 
power of analysis of covariance (ANCOVA) were investigated. 
The data situations included in the study involved two 
groups with one covariate and one criterion, with varying 
equal and unequal group sizes, and varying degrees of 
violation of the assumption of homogeneity of regression. fc 

Results indicated that ANCOVA appeared robust to the 

violation of the assumption of homogeneity of regression 
when group sizes were equal; the technique appeared not 
to be robust for unequal group sizes. For equal group sizes 
and all slope combinations, the empirical alpha levels were 
near the corresponding nominal alpha levels. For unequal 
group sizes and unequal regression slopes, however, large 
discrepancies were observed between the empirical alpha 
levels and the corresponding nominal alpha levels. Results 
also indicated that the power of ANCOVA was not severely 
altered by heterogeneous regression slopes as long as the 
group sizes were equal . 



AN EMPIRICAL INVESTIGATION OF THE EFFECTS 
OF HETEROGENEOUS REGRESSION SLOPES 

IN ANALYSIS OF COVARIANCE 1 
1 . Introduction 
Considerable attention has been centered on the effects 
of violations of the assumptions of analysis of variance 
(ANOVA). The robustness of AN OVA to the violation of 
certain of its assumptions has led to similar questions 

concerning analysis of covariance (ANCOVA) and its assumptions. 
A recent article by Glass et al . (^972) details the work that 
has been done investigating the effi ts of violation of the 
assumptions of ANCOVA. Of particular interest is the 
assumption of homogeneity of regression slopes and the Monte 
Carlo study by Peckhani (1968) that investigated the effects 
of heterogeneous regression slopes. Peckham investigated 
the goodness - of - f i t of the empirical ANCOVA F distribution 
to the theoretical F-distribut ion under violation of the 
homogeneity of regression assumption. He varied *he number 
of treatment groups and the number of subjects per treatment 
group for different sets of heterogeneous regression slopes 
and compared the actual a level with the nominal a level 
under null conditions. All other assumptions of parametric 



ANCOVA were met . Peckham found that the actual # level was 
almost always slightly less than the nominal a level which 
resulted in a conservative test. He found that there was 
goodness-of ~f it for regression slopes differing as much as 
.3 and .7 with the test tending to be more conservative as 
the heterogeneity of the slopes increased. His conclusion 
was that parametric analysis of covariance was robust to all 
but extreme violations of th*e assumption of homogeneity of 
regression . 

Peckham observed that the actual rate of Type I error 
rate was reduced, but he did not investigate the resulting 
effect this mi^ht have on the power of ANCOVA; and, as 
pointed out by Glass et al. (1972), "This could very well 
be the crucial issue [p. 279]." Furthermore, the effects 
of unequal group sizes and, according to Glass et al. (1972), 
the effects of a random covariate have yet to be investigated. 

The purpose of the present study was to in ;tigate the 
effects of violation of the assumption of homogeneity of 
regression upon the Type I error rate and the power of 
ANCOVA. The data situations included a random covariate 
wi th both equa 1 and unequa 1 group sizes. 
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Data Situat i ons Examined 

This study simulated a two group experimental situation 
with one criterion and one covariate. The group sizes used 
were 10,10; 20,20; 30,30; 10,20; t0,30; 20,30; 20,10; 30,10; 
and 30,20. Table 1 contains a listing of the slope combi- 
nations used. Following Peckham (1968) , an attempt was made 
to include slope combinations tnat might be encountered in 
actual research situations. Nominal significance levels of 
.10, .05, *02, and .01 were used for the comparisons of 
actual Type I error rates with nominal Type I error rates. 
Significance levels of .10, .05, .02, .01, .005, and .001 
were used for investigating the effects on the pows?r of 
ANOOVA . 



Insert Table 1 about here 



Each pair of group sizes was combined with each pair 

of slope sizes resulting in 225 different goodness-of -f it 

testing situations. Power investigations were made for 

each pair of equal group sizes in combination with each 

2 

pair of slopes resulting in 75 different power runs . 



Random Number Generation Procedure 

The random number generator used in this study was 
RANDN (Math-Pack, 1970). RANDN is designed to produce a set 
of N pseudo -random numbers which ^re normally distributed 
with specified mean and standard deviation. A check was 
made of, the randomness and normality of 100 samples of 



size 2C generated by RANDN. The one-sample runs test 
(Siegel, 1956) was used to check the randomness of the num- 
bers, and the Kolmogorov-Smirnov one-sample goodness-of -f it 
test (Siegel, 1956) was used to check the normality of the 
samples. Both tests were run using a level of significance 
of .05, and both yielded four rejections out of the 100 
tests made. 

1 The veneration of the slopes within each treatment 
group was accomplished by means of a procedure used by Knapp 
and Swoyer (1967), involving the following theorem: "Let X 
and W be two independent random normal variables with zero 



correlation between X and Y # p x ^» is equal to a [p. 393] . " 
So, using RANDN and the formula listed above, a bivariate 
set of data can be generated with a given slope by first 
calling RANDN to generate X with a mean of zero and a 
standard deviation of one, then calling RANDN again to 
cenerate W with a mean of zero and a standard deviation of 



mean and unit variance. 



Then if Y = aX + 




the 



one, and finally using the formula to generate Y such that 
the correlation between X and Y is equal to a. Since both 
X and Y have unit standard deviations, the slope will equal 
the correlation coefficient , The above process was used to 
generate the data for each of the-two groups with the slope 
combinations listed earlier. For the power runs, unequal 
means were generated by adding .25 to every value of the 
criterion in group one and subtracting .25 from every value 
of the criterion in group two. Thus, a moderate difference 
between group means of .5 standard deviations was induced to 
provide the power comparisons. 
Goodnes s -o f -f it Procedure 

In order to investigate the goodness-of -f it of ANCOVA 
to the corresponding theoretical F distribution under 
violation of the assumption of homogeneity of regression, 
samples were generated from populations which had equal 
means anc unequal regression slopes. ANCOVA was applied 
to the data, obtaining the sample F ratio. The above 
process was repeated 3,000 times for each data situation, 
thus generating an empirical sampling distribution for 
each data situation. 

The goodness-of -f it of each empirical sampling 
distribution to the theoretical was tested using the 
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Kolmogorov -Smir no v one -sample goodness-of -f it test. For the 
purpose of constructing the cumulative frequency distribution, 
the theoretical distribution was divided into 100 parts of 
one percent each. The 99 F values thus obtained were used 
to construct the cumulative frequency distribution of the 
sampling distribution. The subprogram FISHIN (Stat-Pack, 
1969) was called from the University of Maryland program 
library in order to provide zhese percentiles. A signif- 
icance level of .05 was used for these goodness-of ~f it tests. 

In addition to examining the goodness-of -fit under 
violation of the assumption of homogeneity of regression, 
the goodness -of -fi t was also investigated for five sets of 
equal regression slopes in order to provide a check on the 
^htire simulation procedure. 

The goodness-of -f it phase of this study also made it 
possible to investigate the effects of the various data 
situations on Type I error rates. An actual significance 
level for each of four nominal significance levels was 
estimated by determining the proportion of times the test 
statistic exceeded the critical value. These values were 
computed for all the data situations examined in the 
goodness -o f -fit phase of this study . 



P ower Procedure 

The power of ANCOVA under violation of the assumption 
of homogeneity of regression was studi€»d by first generating 
samples from populations which had unequal means and unequal 
regression slopes and then applying the parametric analysis 
of covariance technique to the data, thus obtaining the 
sample F ratio. The obtained F ratio was compared to a 
tabled F value for the specified a levels. The above process 
was repeated 3,000 times for each data situation so that 
relatively stable estimates could be calculated. The 
proportion of times the ANCOVA technique yielded a rejection 
of the null hypothesis of no criterion mean difference was 
computed for each of the specified # levels. This proportion 
yielded an empirical estimate of the power of parametric 
analysis of covariance under each specified assumption 
violation . 

In addition to examining the power under the violation 
of the assumption of homogeneity of regression, powers were 
also computed for five sets of J equal regression slopes in 
order to provide a check on the entire simulation procedure. 

3. Results 

Table 2 presents the results of the goodness-o f -f it 
tests for all group sizes and all sets of regression s lope 3 . 
The symbol A stands for the acceptance of the goodness-of -f i t 



test, and R stands for the rejection of the goodness-of-f it 
test. 

Table 3 prerents, for all group sizes and all sets of 
regression slopes, the empirical Type I error rates corre- 
sponding to the nominal Type I er-aror rates of .10, .05, .02, 
and . 01 . 

Table 4 presents the empirical powers for equal group 
sizes and all regression slopes. For all the power tables 
the decimal point was oir.itted to conserve space. 



Insert Tables 2, 3, and 4 about here 



4. Discussion 

Goodness-of-f it 

According to information presented in Table 2, the 
goodness-of -f i t hypotheses for ANCOVA were accepted in 
all but two of the 60 tests made under violation of the 
assumption of homogeneity of regression with equal group 
sizes. Thus, for the data situations examined, ANCOVA 
appears to be robust to the violation of the assumption 
of homogeneity of regression when group sizej are equal. 
However, the goodness-of-f it hypotheses for ANCOVA were 
rejected in 95 of the 120 tests made under violation of 
the assumption of homogeneity of regression with unequal 
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group sizes. According to these results, for the* data 
situations examined, ANCOVA appears not to be robust to the 
violation of the assumption of homogeneity of regression 
when group sizes are unequal. However, it should be noted 
that when unequal regression slopes were coupled with 
unequal group sizes that were large, suv-h as 20 and 30, 
there was a tendency to accept the goodness-of -f it hypotheses 
when the slopes did not greatly differ. 

From Table 3, it appears that for equal group sizes and 
all slope combinations, the empirical alpha levels for 
ANTOVA were near the corresponding nominal alpha levels. 
It is recognized that g^odness-of-f it tests that lead to 
rejection can be misleading if the lack of fit occurs in 
the central portion of the distribution. However, such was 
not the case in thi s study . Inspection of the data in 
'"able 3 reveals that for unequal group sizes and unequal 
regression slopes, large discrepancies were observed between 
the empirical alpha levels for ANCOVA and the corresponding 
nominal alpha levels. For data situations in which the 
larger group size was coupled with the larger of the two 
regression slopes, the empirical alpha levels were greater 
than the corresponding nominal alpha levels. For data 
situations in which the larger group cize was coupled with 
the smaller of the two regression slopes, the empirical 



alpha levels were less than the corresponding nominal alpha 
levels. These results seem to indicate that if ANCOVA were 
used with unequal group sizes and unequal regression slopes, 
the Type I error rate could be severely altered in a 
predictable direction. For a situation in which the larger 
aroup size is coupled with the larger of the two regression 
slopes , r eject ion of a null hypothes is may result from an 
inflated Type I error rate rather than an actual differenca 
in populations. For a situation in which the larger group 
size is coupled with the smaller of the two regression 
slopes, failure to reject a null hypothesis may result from 
the loss of power associated with a def later alpha. 
Power 

Inspection of the power figures for equal group sizes 
and for equal regression slopes in Table 4 reveals that the 
power procedure appeared to have functioned properly. For 
a given level of group sizes, power levels increased as 
correlations between ccvariate and criterion increased; 
and for a giw-n set of slope combinations, power levels 
increased as group sizes increased . 

Toble 4 is organized to facilitate the comparison of 
power levels of data situations that meet the assumption 
of homogeneity of regression with power levels of data 
situations that violate the assumption of homogeneity of 



regression. For example, with an alpha level of .10, group 
sizes of 30 and 30, and slope combination of .5 and .5, the 
probability of rejecting the false null hypothesis was 
computed to be .716. This power level was determined for 
a data situation in which the assAimption of homogeneity of 
regression was satisfied. The power level immediately to 
the right of .716 represents the probability of rejecting 
the false null hypothesis when the assumption of homogeneity 
of regression has been violated. This value of .711 is the 
proportion of times the false null hypothesis was rejected 
when the population slopes were .4 and .6. The next three 



power levels to the right of .711 represent empirical power 
levels under more extreme violation of the assumption of 
homogeneity of regression. So, i* the power is computed 
assuming equal regression slopes of .5 and .5, the loss in 
power is minimal if the true population regression slopes 
are .4, .6; .3, .7; .2, .8; or . 1, . 9 . 

Similar comparisons for other portions of Table 4 
repeal that there is little or no loss of power when' the 



assumption of homogeneity of regression has been violated. 
So, both the Type I error rate and the power do not seem 
to be severely altered by heterogeneous regression slopes 
as long as the group sizes are equal. 
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One final point needs to be made. As pointed out by 
Bradley (1964), 

. • . the question of tne relative sensitivity of a test to 
violation of its various assumptions [robustness] is fairly 
meaningless unless one is willing-to specify exactly "how 
much 1 ' violation and under exactly what sampling conditions 
(i.e., what sample si7.es, what significance levels, what 
rejection regions, etc.). The robustness of the test 
depends upon the speci fic situation [p. 171]. 
Therefore, the findings of this study will of necessity be 
defined in terms of the specific data situations analyzed. 
While certain tentative conclusions have been drawn here, 
there should be no attempt to generalize beyond the specific 
data situations investigated in this study. Whether or not 
the results observed in this study will hold for other slope 
combinations, other group sizes, more than two groups, etc. 
will have to await further research. 
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Footnotes 

^This pape:: is based on the doctoral dissertation, 

"A M^nte Carlo Comparison of Parametric and Nonparametr ic 

Uses of a Concomitant Variable," by Basil L. Hamilton, 

University of Maryland, College Park, Maryland, 1972. 
2 

All computer programs used in this study were written 
by the author in FORTRAN V. Complete listings are available 
on reques t . 
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Mean Slope 



.5 



Table 1 

Slope Combinations Examined 

#1 #i &x &i &i Pi &i 

i&2 ^2 ^2 $2 &2 &2 $2 

.3 .2 .1 .0 -.1 -.2 -.3 

.3 .4 .5 .6 .7 .8 .9 

.4 .3 .2 .1 .0 -.1 

.4 .5 .6 .7 .8 .9 

.5 .4 .3 .2 .1 

.5 .6 .7 .8 .9 

.6 .5 .4 .3 

.6 .7 .8 .9 

.7 .6 .5 

.8 .9 
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T»bl« 3 

fcaplrlcal Vyp« 1 error lUt«a for An»l/»t« of Cov«rl*nc« for 
Different Croup Site* and Slop* Coafelnttlona 
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